Impact of signal-to-noise ratio and contrast definition on the sensitivity assessment and benchmarking of fluorescence molecular imaging systems

Abstract. Significance Standardization of fluorescence molecular imaging (FMI) is critical for ensuring quality control in guiding surgical procedures. To accurately evaluate system performance, two metrics, the signal-to-noise ratio (SNR) and contrast, are widely employed. However, there is currently no consensus on how these metrics can be computed. Aim We aim to examine the impact of SNR and contrast definitions on the performance assessment of FMI systems. Approach We quantified the SNR and contrast of six near-infrared FMI systems by imaging a multi-parametric phantom. Based on approaches commonly used in the literature, we quantified seven SNRs and four contrast values considering different background regions and/or formulas. Then, we calculated benchmarking (BM) scores and respective rank values for each system. Results We show that the performance assessment of an FMI system changes depending on the background locations and the applied quantification method. For a single system, the different metrics can vary up to ∼35  dB (SNR), ∼8.65  a.u. (contrast), and ∼0.67  a.u. (BM score). Conclusions The definition of precise guidelines for FMI performance assessment is imperative to ensure successful clinical translation of the technology. Such guidelines can also enable quality control for the already clinically approved indocyanine green-based fluorescence image-guided surgery.


Introduction
Fluorescence molecular imaging (FMI) has made great advances in clinical translation over the last few years. 1 Driven by these advances, technologies at the forefront of the field are evolving rapidly, particularly in the areas of device design, fluorescent agents, image processing algorithms, and performance assessment metrics. 2 Consequently, the number of imaging devices and their applications is increasing.
Moreover, following the first-in-human application of FMI in 2011 by van Dam et al., 3 numerous clinical studies have been completed or are currently ongoing.A major outcome of all this activity is the recent approvals by the US Food and Drug Administration (FDA) of ∼20 fluorescence-guided clinical imaging systems 4 as well as 3 tracers for surgical guidance: (1) 5-aminolevulinic acid (5-ALA/Gleolan®; Photonamic GmbH and Co., KG, Pinneberg, Germany) for use as an intra-operative optical imaging agent in patients with suspected high-grade gliomas, 5 (2) hexaminolevulinate (HAL, available as Hexvix, Photocure ASA, Oslo, Norway, and Cysview Photocure Inc., Princeton, New Jersey, United States) for use in non-muscle-invasive bladder cancer, 6 and (3) pafolacianine (Cytalux, On Target Laboratories LLC, West Lafayette, Indiana, United States) for intraoperative imaging of folate receptorpositive ovarian and lung cancers. 7,8All this activity has highlighted the need for better and user-independent standardization procedures that would allow for system characterization, performance monitoring, data referencing, and comparison, even among markedly different systems.This is, also, very relevant to the fluorescent image-guided surgery (FIGS), given the FDA clearance of multiple FIGS devices for imaging with indocyanine green (ICG) and other contrast agents. 92][13][14][15][16][17][18][19] Thus far, methods and reference targets for system evaluation and comparison have been developed on an individual basis, but a universal cross-platform metric for image fidelity evaluation has yet to be developed. 16urrently, the sensitivity of FMI systems is assessed mostly using the signal-to-noise ratio (SNR) and/or contrast metrics. 2,12,17,202][23][24] For example, Chen et al. 22 and Hoogstins et al. 21reported that background estimation significantly affected quantification results for bulk-stained tissue fluorescence imaging and intraoperative/ex vivo fluorescence imaging, respectively, using metrics including SNR, signal-to-background ratio (SBR), and contrast-to-noise ratio (CNR).Widen et al. 23 demonstrated the impact of region of interest (ROI) sizes on overall signal and mean fluorescence intensity by analyzing fluorescent probes in animal experiments.Dijkhuis et al. 24 also demonstrated the effect of manually selected ROI in fluorescent data analysis and proposed semi-automatic methods for objective assessment of fluorescent signals in resected tissue.In view of the theoretical effect described above, Azargoshasb et al. 25 quantified how fluorescent SBR influences the robotic surgical performance of participants (n ¼ 16) during an exercise with a custom grid phantom.On the other hand, Palma-Chavez et al. 26 reported 15 different SNRs and five contrast formulas that are currently used in the field of optoacoustics, indicating that the lack of consensus is not only limited to FMI applications.The plethora of background definitions, as well as the different quantification formulas used across multiple studies, emphasize the importance of reaching a wide consensus for performance assessment and quality control of FMI systems.
Indeed, despite the fact that SNR and contrast are the most commonly used metrics for the sensitivity assessment of various systems, 1,26 there are only a few studies comparing different FIGS systems, most of which are optimized for ICG imaging. 9,19,27In addition, the formulas used to calculate SNR and contrast and methods for evaluating background ROIs vary across different studies.In a recent study, LaRochelle et al. 28 demonstrated the influence of background definition in SBR, SNR, CNR, and contrast-to-variability ratio through measurements on anthropomorphic three-dimensional (3D)-printed phantoms.However, to the best of our knowledge, there is no study quantifying the effect of the combined variation (ROIs and metrics formulas) on performance assessment.An in-depth testing and evaluation of current strategies are crucial to raise community awareness of existing limitations, to spur effective development of the technology, and to set the performance limits that are required for regulatory approvals.
Building on the assumption that the SNR and contrast metrics depend on the selection of background ROIs and quantification formulas, herein, for the first time, we systematically investigate and showcase this dependence with regard to the sensitivity assessment of markedly different FMI systems.
In specifics, using six near-infrared FMI systems, we captured fluorescence images of a composite rigid phantom previously developed by our group. 11,18,19We then assessed the sensitivity 19 of those systems using six previously published formulas for SNR and contrast 17,[29][30][31][32][33] and two background locations.Moreover, based on these metrics, we quantified the corresponding benchmarking (BM) scores, 19 and the systems were ranked based on these scores.
Recently, we called attention to the need for a commonly accepted phantom to promote good imaging practices during the development of FMI systems or their use in clinics. 1 We now pinpoint additional needs to consistently define ROIs and use common quantification formulas for SNR and contrast.Answering these needs will enable consistency, allow data comparison and referencing, and advance the quality and performance of FMI systems.These improvements will promote wide acceptance and usage of FMI as a tool for interventional and endoscopic procedures.

FMISystems
For this study, we used six fluorescence imaging systems distributed in different labs in the United States and Europe.The main specifications of each system, as well as the adopted phantom imaging protocols, are summarized in Table 1, while the corresponding system schematics are presented in Fig. 1.All measurements were conducted in darkness to eliminate the influence of ambient light on the results.
Mob is a mobile phone-based near-infrared fluorescence (NIRF) imaging system previously, 34 where its spectral sensitivity was documented.It involves a 1W 785-nm laser diode, an 800-nm short-pass excitation filter (84-729, Edmund Optics, Barrington, New Jersey, United States), and a long-pass emission filter with a cutoff wavelength at 825 nm (86-078, Edmund Optics) for the detection.The phone camera is based on an 8-bit complementary metal oxide semiconductor (CMOS) sensor with an f/2.4 aperture lens (Eigen Imaging, Inc., San Diego, California, United States) and a near-infrared blocking filter, which was removed during this study.NIRF I is a custom benchtop NIRF imaging system 35 with a light-emitting diode (M780L3, Thorlabs, Inc., Newton, New Jersey, United States) centered at 780 nm and power of 200 mW.The same optical filters used with the Mob system were also used in the NIRF I imaging system.A 16-bit charge-coupled device camera (Alta U2000, Apogee Imaging Systems, Roseville, California, United States) coupled with a zoom lens (7-mm focal length, f/3.9, Tamron, Commack, New York, United States) was used for the detection of the emitted fluorescence.
NIRF II is an updated version of the NIRF I imaging system.Its main improvement is the replacement of the imaging sensor with the more sensitive Kodak KAI-2020M (Image Sensor Solutions Eastman Kodak Company, Rochester, New York, United States), while fluorescence was induced by a laser diode at 785 nm and 1W power, instead of the light-emitting diode present in NIRF I system.
Solaris is an open-air commercially available fluorescence imaging system by PerkinElmer (Waltham, Massachusetts, United States).The Solaris system is designed for research applications, including preclinical studies for advanced molecular-guided surgery, and drug efficacy and safety measurements.
RawFl is a custom-built setup 36 with a filtered 760-nm laser diode (LDX Optronics, Maryville, Tennessee, United States) light source, a 16-bit scientific complementary metal oxide semiconductor (sCMOS) camera (pco.edge5.5, PCO AG, Kelheim, Germany) as a detector and polarizers (PPL05C; Moxtek, Orem, Utah, United States) for minimizing the contribution from specular reflections at the surface of the sample.

Standardization Phantom
The composite phantom shown in Fig. 2(a) 19 was used to quantify the SNR and contrast from images acquired by the six systems.The application of the phantom as a fluorescence standard for performance assessment, quality control, and comparison of markedly different systems through a single image has been described in detail in previous studies. 11,18,19In the current study, however, the SNR and contrast were evaluated only on the "sensitivity versus depth" region of the phantom [see Fig. 2(a)].This region includes (1) the transparent polyurethane (WC-783 A/B, BJB Enterprises, Tustin, California, United States) matrix base, with 0.00875 mg/g alcohol-soluble nigrosin (Sigma Aldrich, St. Louis, Missouri, United States) and 1.5 mg/g TiO 2 nanoparticles (titanium IVoxide; Sigma Aldrich) for mimicking absorption and scattering, and (2) nine equally sized circular wells, made of the same polyurethane base with 20 μg∕g bovine hemin (≥ 90% pure; Sigma Aldrich) and 0.66 mg/g TiO 2 for absorption and scattering and 10-nM organic quantum dots (Qdot 800 ITK, Thermofisher Scientific, Waltham, Massachusetts, United States) for fluorescence.As shown in Fig. 2(a), the nine wells were embedded into the phantom matrix at distances of 0.2, 0.4, 0.6, 0.8, 1.0, 1.33, 1.66, 2.0, and 3.0 mm, respectively, from the phantom's top surface.

Data Processing
The sensitivity versus depth phantom region [Fig.2(a)] was extracted from the fluorescence images acquired by each system, and the SNR and contrast metrics were quantified by adopting the formulas in Table 2.
First, all images of the phantom wells from the region sensitivity versus depth were converted into binary images using the MATLAB function "imbinarize," with the default option of thresholding using the Otsu method (MathWorks, Natick, Massachusetts, United States), and the location and radius of each well were obtained using the "imfindcircles" function.The extracted wells were then adjusted to match the size and location of the phantom wells based on the phantom design template, which ensured all wells preserved the same size within an image, regardless of the per-well fluorescence intensity distribution.Using this information, one mask was created to extract the average fluorescence intensity and standard deviation values from each well.A second mask, consisting of (i) the annuli between each well and concentric to the wells' circles with a 40% larger radius (termed ROI b1) and (ii) a well-sized circular area in the non-fluorescent region of the phantom (termed ROI b2), was also created to quantify the average intensity and  1 for the description of each system).
corresponding standard deviation values from the background ROIs (Fig. 2).The ROI b1 is adjacent to the wells that produce fluorescence signal, where fluorescence leakage to the neighboring phantom areas influences the ROI's intensity values.This is frequently adopted as a strategy for background definition in multiple studies. 9,28The second ROI, b2, is located far from fluorescent wells and thus is not affected by fluorescence leakage.This is another frequently adopted definition of background, especially for studies where autofluorescence or diffusion is strong in the proximity of the target. 13o investigate the impact of chosen ROIs and quantification formulas (Table 2) on the BM of FMI systems, we calculated BM scores for each system as derived from the sensitivity versus depth phantom region using the method previously described. 19Briefly, the BM scores were defined as E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 1 ; 1 1 4 ; 3 0 9 BM ¼ sMAPE∕N; ( where sMAPE is the symmetric mean absolute percentage error of the SNR and contrast metrics that have been quantified for the various formulas of Table 2 and for the two background regions shown in Fig. 2(a).The sMAPE is calculated as ; t e m p : i n t r a l i n k -; e 0 0 2 ; 1 1 4 ; where nis the number of phantom wells included in the metrics' evaluation (n ¼ 9), X i is the value of the metric result (i.e., SNR or contrast), and Y i is the reference value.For the BM score quantification, we considered normal signal distributions, according to which a measurement is assumed to present 95% confidence if the signal is twofold the noise level.This results in reference values of 6 dB for SNR, 0.33 for Michelson contrast, and 1 for Weber contrast. 19ince the scope of this work is to assess how the SNR and contrast change depending on the application of different formulas and/or ROIs, all data processing was implemented on single images of the phantom acquired by the six systems.The repeatability and error analysis of the quantification of those two metrics have been recently reported by our group elsewhere. 39ble 2 Formulas for calculating SNR and contrast.Employing the six FMI systems described in Table 1, we imaged the composite phantom of Fig. 2(a) and isolated the sensitivity versus depth region from the acquired images, as shown in Fig. 2(b).As expected, the markedly different systems yield subjectively different images from the same area of the same phantom, which highlights the importance and need for FMI standardization to ensure a consistently high degree of performance and facilitate clinical translation.
The ROIs used for the quantification of SNR and contrast are shown in the top-right inset of Fig. 2(a).The arrows point out the two areas (b1 and b2) used for background calculation.These locations were chosen based on different studies assessing the performance of FMI systems 13,28 and according to phantom constituents and geometry.
Figures 3 and 4 illustrate the calculated SNR and contrast metrics, respectively, as functions of depth.The results obtained using SNR 1 and SNR 2 show the same trend for all systems, but not equivalent values [Fig.3(b)].Moreover, the results obtained using SNR 3 and SNR 4 not only differ from SNR 1 and SNR 2 but are also influenced by the chosen background area.For instance, when comparing Mob and NIRF I for SNR b1 2 , it is evident from Fig. 3(b) that the NIRF I system has a greater SNR than the Mob system.However, a comparison of the NIRF I system using SNR b1 3 with the Mob system using SNR b1 2 yields the opposite conclusion.Figure 4 demonstrates the results of the contrast metrics with respect to the applied formula (i.e., C M -Michelson contrast and C W -Weber contrast) and considered background ROI.The trends for both C M and C W metrics are similar for each system when the same background values are considered (i.e., b1 or b2 for both formulas).Conversely, when comparing the trends observed in C M under the two background values, the background influence on the quantification of the contrast metrics becomes evident [Fig.4(b)].For example, the Mob system has a higher contrast than the RawFl system when the Michelson contrast is applied under the b1 background for both systems.This is not true, however, when the Michelson contrast is used under b1 for the Mob system and b2 for the RawFl system.In that case, the RawFl system has a higher contrast than the Mob one [see Fig. 4(b)].The influence of the applied formula and background ROI shown in Fig. 3 for SNR and Fig. 4 for contrast becomes even stronger when both metrics are combined to assess the performance of different FMI systems.Figure 5 depicts the ranking of the six systems used in this study based on the corresponding BM scores, which were calculated from the SNR and contrast metrics.Figure 5(a) illustrates the effect of combining the different formulas and background locations on the quantification of the BM scores per system.Moreover, Figs.5(b)-5(e) demonstrate exemplary BM scores for each system as selected from the four marked squares in Fig. 5(a).The four squares were selected after a visual inspection of the map in Fig. 5(a) to showcase the variability in the quantified BM scores.As can be seen, the BM scores not only have different values, but also their trend is different per combination of formulas and background ROIs.This trend becomes clear in Fig. 5(f), where the systems' ranking (i.e., 1-worst through 6-best) is shown for the various BM scores of Figs.5(b)-5(e).For example, the hybrid system's rank is superior to Solaris' rank if their BM scores result from the combination of SNR

Discussion and Conclusion
In the current work, through the comparison of six near-infrared FMI systems, we showed that the assessment of system performance and standardization via SNR and contrast is highly dependent on the definition of background ROI and the formulas used.This proves the need for careful attention to test a method's clinical relevance, as well as consistency in defining metrics for objective, quantitative assessment of FMI system performance.We used fluorescence data from the sensitivity versus depth areas of a multiparametric phantom to quantify SNR and contrast by means of different formulas obtained from the literature (Table 2).It was demonstrated that resultant SNR values can be affected by both the selected background location and the formulas applied (Fig. 3).In the case of contrast values, resultant trends appear similar for both Michelson and Weber formulas (C M and C W in Table 2), but the employed background ROI is still observed to impact the trends.Indeed, as we show in Fig. 4, the contrast (C W ) for the hybrid system changes by a factor of 8.3 depending on the background, while for the Mob system by 2.9.The dependence of the applied formula and/or background becomes more evident for SNR, where the Mob system shows a variation by a factor of 19.6 in the SNR estimation.This indicates a pressing need for common quantification formulas for SNR and contrast and consistent ROI definition for both signal and background.All measurements in this study were conducted in darkness to minimize ambient illumination that would further complicate the quantification of SNR and contrast.However, illumination is another critical factor that must be accounted for when darkness is not possible.One way to address this challenge is by acquiring a "dark" image with the excitation sources turned off and subsequently subtracting that image from the fluorescence image.This step should be performed before quantifying and reporting any performance assessment and quality control metrics.Meeting these requirements is crucial to achieving reliable results and standardization guidelines for FMI. 2,14Having this internal consistency during the development of FMI systems will lead to the establishment of international consensus across the field and will contribute to the widespread acceptance and use of FMI.
Our goal, however, was not only to assess the performance of each system in different SNRs and contrast definitions but also to show how these definitions affect the comparison of markedly different systems.The results of our contrast and SNR calculations were translated into BM scores and then to rank values.This analysis revealed the dependence of the ranking on the definition of background ROIs or the adopted formulas (Fig. 5).For example, the rank value for the Solaris system was lower than the corresponding values for the NIRF II and the hybrid systems if the performance assessment was based on the Michelson contrast and SNR 2 formula with background defined as b1.However, the Solaris system ranks higher than the NIRF II and the hybrid systems when SNR is evaluated as SNR 1 and contrast through the Weber formula with the b2.This inconsistency in the determinants of the metrics for system evaluation can affect  the development and comparison of systems and ultimately the design and efficacy of clinical or pre-clinical studies.In a recent report, the American Association of Physicists in Medicine (AAPM) proposed SNR 4 and C M , as metrics for the performance assessment of fluorescence imaging systems. 17Moreover, the suggested background region proposed in these guidelines for the estimation of SNR corresponds to a region with the same optical properties as the interrogated wells, but without fluorescent dye.This corresponds to the ROI b 2 in our study since the wells are gradually covered with the phantom matrix material which has no fluorescent dye.On the other hand, in the AAPM study, the contrast is associated with the resolution of a system and not the signal contrast as employed herein.Thus, although there is agreement in the SNR definition (SNR b1 4 ) between the AAPM and our study, we additionally employed the contrast as a means of sensitivity assessment.Nevertheless, these recommendations represent a promising initial step toward establishing a widely accepted protocol for standardizing FMI systems, thereby addressing the inconsistencies demonstrated herein.
Similar limitations for the quantification of SNR and contrast have also been reported during the use of FMI systems in pre-clinical and clinical applications.For example, LaRochelle et al. 28 discussed the variability of the methods used for reporting the quantitative sensitivity metrics using 3D anthropomorphic phantoms with incorporated NIR fluorescent tumor parts.On the other hand, Hoogstins et al. 21used data from both animal and human studies with multiple fluorescence tracers to show that background noise and background selection have a significant influence on the quantification of SBR and contrast-to-background ratio.Similarly, Azargoshasb et al. 25 showed that SBR quantification can impact the surgical discrimination of fluorescence signals, highlighting the importance of the applied quantification approach in intraoperative decision-making.Herein, we present, to the best of our knowledge, the first study that showcases not only how the adopted formulas and the used background affect the performance assessment of an FMI system but also how the lack of consensus on quantification methods of SNR and contrast can result to misleading interpretation of system comparison measurements.
Moreover, for the quantification of the BM scores, we assumed normal signal distributions, according to which a measurement represents 95% confidence when its value is twice the magnitude of the noise level. 19Thus, the reference threshold values applied here are user-independent in comparison to another value commonly used in fluorescence imaging, the Rose criterion. 17he Rose criterion method also sets a limit of detection for fluorescence imaging for which the CNR values must be greater than 3 to 5. 40 However, the range of a particular threshold value varies from study to study 35,41,42 and depends on several parameters such as object shape, edge sharpness, viewing distance, and observer experience.Besides the parameters affecting the threshold value, Rose's studies were intended for electronic imaging systems (i.e., photography, television, and optical and visual systems) 43 and were focused on human perception of signal detectability. 44However, threshold values that are constrained by aspects of the human visual system might no longer be relevant with the advent of artificial intelligence (AI) imaging and signal processing.6][47] The criterion adopted herein follows a more simplistic statistical approach that evaluates system performance without depending on human perception and thus is more relevant for assessing the detection limits of FMI systems.
The findings of this study are also relevant to existing ICG-based FIGS systems.Similar to FMI, most FIGS system sensitivity assessment and quality control approaches are still based on the quantification of SNR and contrast metrics.However, the quantification methods for these metrics still represent a major limitation factor for cross-platform system comparisons and affect the design and/or repeatability of preclinical or clinical trials.Moreover, consistency in quantification and reporting of the various performance assessment metrics is especially important for FIGS systems, as no established quality control protocols currently exist despite the wide clinical use of such systems.The quantitative assessment of the system performance presented herein advances the current standardization strategies, which is critical for the further development of this technology and for establishing the performance limits that are a prerequisite for regulatory approvals.
Finally, similar challenges in the quantification of SNR and contrast are present in other optical technologies that are currently under investigation.For example, Palma-Chavez et al. 26 showcased variability in SNR and contrast quantification methods within the field of optoacoustics.Fluorescence lifetime imaging is another emerging and very promising technology that also lacks consensus in the quantification of SNR, despite its frequent use in assessing the reliability of lifetime measurements.Under appropriate modifications, our study can also be adapted for such technologies, thereby contributing to the development of performance assessment and quality control protocols for imaging methods beyond FMI and FIGS.

Fig. 2
Fig. 2 Sensitivity versus depth phantom region.(a) An illustration of the composite phantom used in this study, with the sensitivity versus depth wells highlighted and enlarged.Arrows denote two areas (b1 and b2) used as background regions.The depth of the phantom wells (bottom left, Dx where x ¼ a; b; c: : : ) indicates the distance from the top surface of the phantom to each fluorescent inclusion.The concentrations of different constituents are the same for all inclusions.Qdots, quantum dots for fluorescence; Hemin, bovine hemin; and TiO 2 , nanoparticles (see Sec. 2.1).(b) Fluorescence images normalized to their corresponding maxima as acquired by the six systems employed in the study (see Table1for the description of each system).

30 SNR b2 2 SNR b1 3 SNR 31 SNR b2 3 SNR b1 4 SNR
of photons on the detector; σ-the noise associated with the detector (i.e., standard deviation) foreground signal pixel intensity; N-mean background noise pixel intensity ¼ μ S−N σ S μ S−N -mean signal after background subtraction; σ S -standard deviation of the signal ¼ S−N σ N S-mean signal pixel intensity; N-mean background noise pixel intensity, σ N -background standard deviation 17 I max −I min I max þI min I max ; I min -maximum pixel intensity and minimum background pixel intensity,

Fig. 3
Fig. 3 Dependence of SNR on the two background locations shown in Fig. 2(a) and/or the quantification formulas of Table 2 for different FMI systems.(a) SNR values for all systems at each depth.SNR 1 shows the same behavior for each system as a function of depth.SNR 2 shows a similar trend to SNR 1 for all systems, regardless of the background employed.SNR 3 and SNR 4 show different trends compared with SNR 1 and SNR 2 , depending on the background.(b) SNR values of the phantom well with depth = 1 mm for all systems.The values correspond to the dashed area highlighted in panel (a).

Fig. 4
Fig. 4 Dependence of contrast on the two background locations shown in Fig. 2(a) and/or the quantification formulas of Table 2 for different FMI systems.(a) The contrast metric results for all systems at each depth.C M and C W show similar trends when either b1 or b2 is employed for both calculations.(b) Contrast results for the phantom well with depth = 1 mm for all systems.The values correspond to the dashed area highlighted in panel (a).

Fig. 5
Fig. 5 BM scores calculated according to Gorpas et al. 19 for each system.(a) Map of the BM scores quantified using different SNRs and contrast (C) formulas (see Table 2) and two different backgrounds [see Fig. 2(a)].The squares marked with numbers 1, 2, 3, and 4 correspond to the representative graphs of BM scores in panel (b) for square 1, SNR b1 2 and C b1 M ; (c) for square 2, SNR 1 ; (d) for square 3, SNR b1 3 ; and (e) for square 4, SNR b2 4 .(f) The rank of each system as a result of the BM scores for all squares of panel (a).
Fig. 5 BM scores calculated according to Gorpas et al. 19 for each system.(a) Map of the BM scores quantified using different SNRs and contrast (C) formulas (see Table 2) and two different backgrounds [see Fig. 2(a)].The squares marked with numbers 1, 2, 3, and 4 correspond to the representative graphs of BM scores in panel (b) for square 1, SNR b1 2 and C b1 M ; (c) for square 2, SNR 1 ; (d) for square 3, SNR b1 3 ; and (e) for square 4, SNR b2 4 .(f) The rank of each system as a result of the BM scores for all squares of panel (a).

Table 1
FMI systems used in this study and the corresponding imaging protocols.
a At the phantom surface.